Applying FAIR4RS to your projects

Sylvia Whittle (she/her)

Our project in March 2022

  • On GitHub
  • Two developers
  • Changes made by one would break the other’s code
  • Difficult to install (sometimes impossible!)
  • Deprecated python 2.7
  • To change a variable, you had to open the code

Our project in March 2022: installation

  • Uninstall all python, gwydidon, pygobject, pycairo, pygtk installations
  • Delete all caches of the above softwares
  • Install Anaconda 32 bit
  • Install python 2.7
  • Install pycharm (register for an account if necessary)
  • Install Gwyddion (From an unfamiliar website)
  • Download a set of additional files from Google drive,

Our project in March 2022: installation continued

  • Import the environment from the gwyconda.yml file.
  • Follow some images to determine which checkboxes to select.
  • Locate your python environment
  • Install the PyGTK2 packages:
    • Install PyGTK
    • Install PyCairo
    • Install PyGObject

Our project in March 2022: installation continued…

  • Manually add the paths for these into Anaconda

  • Change the Gwyconda environment directory to the bin folder in Gwyddion

  • Open a new project and set the interpreter to Gwyconda

  • Create a python file

  • Append the path of the bin folder in Gwyddion

  • Ignore all runtime warnings

  • (Go back to the start because something went wrong in the installation)

So what did we do?

  • We asked for help 🙂
  • Contacted the Research Software Engineering team
  • Together with the help of the RSE team, we started incorporating FAIR4RS principles into our project

Updated GitHub repository: process

  • Added a welcoming README file
  • Ensured the licensing was clear
  • Added a code of conduct
  • Added a contributing guide

Effort: 1 day

Updated GitHub repository: benefits

  • Friendly landing page for new users
  • Clear licensing so people know that the code is free, open source, and can be used for research!
  • Code of conduct to help people feel safe and welcome
  • Contributing guide to help people get involved

Updating to python 3: process

  • Python 2.7 was deprecated in January 2020 🪦
  • Started from scratch in Python 3
  • Ditched some dependencies, updated others
  • Wrote our own code to replace some of the dependencies that were no longer compatible

Effort: 4 months

Updating to python 3: benefits

  • Code was more stable
  • More people could use it
  • Up-to-date python features and libraries
  • Online tutorials & examples were relevant

Improving installation: process

  • Removed the need to install dependencies separately by writing our own code to replace them, or using python 3 libraries
  • Published the code to PyPI so it can be installed by anyone

Effort: 1 day once the code was ready

Improving installation

New installation procedure:

pip install topostats

Improving installation: benefits

  • Friendlier installation means more users
  • Onboarding is so much faster
  • Now works on all computers!
  • Time is saved with every new user, and new member of our research team

Configuration file: process

  • Removed all hard-coded parameters from the code
  • Created a configuration file in YAML format
  • Wrote code to load the configuration file
  • Wrote code to check that the configuration file is valid and makes sense

Effort: 1 week for basic functionality

Configuration file: benefits

  • Easy way to group parameters into one place
  • Users don’t have to open the code
  • Configurations saved with the output files for reproducibility

Documentation: process

  • Wrote a README file to explain the project
  • Wrote documentation in the form of docstrings for every function, class and file
  • Wrote a tutorial to help users get started
  • Set up automatic documentation generation and hosting using Sphinx (involved, but worth it!)

Effort: 1 week

Documentation: benefits

  • Each function, class and file is explained for developers and users
  • Documentation available online as a website
  • Automatically generated & updated (no effort once set up!)
  • Eliminates the time I spend wondering what a function I wrote 6 months ago does!

Code linting: process

  • Installed an automatic formatter (black for python)
  • Ran the formatter on the code, which automatically re-structures the code to match best practices

Effort: 1 hour

  • Installed a linter (flake8 for python)
  • Ran the linter on the code, which checks that the code is written in a way that adheres to best practices
  • Fix the many errors and warnings

Effort: 1 month

Code linting: benefits

  • Everyone speaks the same coding “dialect”
  • Code is easier to read and understand
  • Bugs are harder to introduce when following best practices
  • Reinforces good coding habits

Versioning: process

  • Set up versioning on GitHub
  • Set up automatic versioning on PyPI
  • Automatic documentation versioning & generation using Sphinx
  • Automatically publish new versions to ORDA

Effort: 1 day for basic functionality

Versioning: benefits

  • Development versions are on GitHub
  • Stable versions are released to PyPI
  • Documentation is versioned too!
  • Every version of the project is archived, usable and citable

Testing: process

  • Wrote tests for every function, class and file
  • Set up automatic testing on GitHub using GitHub Actions

Effort: Months, ongoing

Testing: benefits

  • Difficult to begin with
  • Adds time to development
  • More than makes up for it when writing new features
  • Confidence in the code that it does what it is was written to do! (and continues to do so!) (and nothing else!!) (on all types of computers!!!)

Our project in 2023

  • 70% test coverage
  • 95% ish documentation coverage
  • 250 ish downloads per month on PyPI
  • Collaborators from around the world
  • 8+ projects in our research group relying on our code

Reassurances

  • We did all this incrementally over a long time, with the help of the RSE team

  • You don’t have to do it all at once

  • You don’t have to do it all yourself

  • You don’t have to do it perfectly

  • Small changes make a big difference!